Robotic teleoperation is a key technology for a wide variety of applications. It allows sending robots instead of humans in remote, possibly dangerous locations while still using the human brain with its enormous knowledge and creativity, especially for solving unexpected problems. A main challenge in teleoperation consists of providing enough feedback to the human operator for situation awareness and thus create full immersion, as well as offering the operator suitable control interfaces to achieve efficient and robust task fulfillment. We present a bimanual telemanipulation system consisting of an anthropomorphic avatar robot and an operator station providing force and haptic feedback to the human operator. The avatar arms are controlled in Cartesian space with a direct mapping of the operator movements. The measured forces and torques on the avatar side are haptically displayed to the operator. We developed a predictive avatar model for limit avoidance which runs on the operator side, ensuring low latency. The system was successfully evaluated during the ANA Avatar XPRIZE competition semifinals. In addition, we performed in lab experiments and carried out a small user study with mostly untrained operators.

Earthquakes, fire, and floods often cause structural collapses of buildings. The inspection of damaged buildings poses a high risk for emergency forces or is even impossible, though. We present three recent selected missions of the Robotics Task Force of the German Rescue Robotics Center, where both ground and aerial robots were used to explore destroyed buildings. We describe and reflect the missions as well as the lessons learned that have resulted from them. In order to make robots from research laboratories fit for real operations, realistic test environments were set up for outdoor and indoor use and tested in regular exercises by researchers and emergency forces. Based on this experience, the robots and their control software were significantly improved. Furthermore, top teams of researchers and first responders were formed, each with realistic assessments of the operational and practical suitability of robotic systems.

Dexterous manipulation with anthropomorphic robot hands remains a challenging problem in robotics because of the high-dimensional state and action spaces and complex contacts. Nevertheless, skillful closed-loop manipulation is required to enable humanoid robots to operate in unstructured real-world environments. Reinforcement learning (RL) has traditionally imposed enormous interaction data requirements for optimizing such complex control problems. We introduce a new framework that leverages recent advances in GPU-based simulation along with the strength of imitation learning in guiding policy search towards promising behaviors to make RL training feasible in these domains. To this end, we present an immersive virtual reality teleoperation interface designed for interactive human-like manipulation on contact rich tasks and a suite of manipulation environments inspired by tasks of daily living. Finally, we demonstrate the complementary strengths of massively parallel RL and imitation learning, yielding robust and natural behaviors. Videos of trained policies, our source code, and the collected demonstration datasets are available at https://maltemosbach.github.io/interactive_ human_like_manipulation/.

多摄像机系统的校准，即确定相机之间的相对姿势，是计算机视觉和机器人技术中许多任务的先决条件。通常使用使用棋盘校准目标的离线方法来实现摄像头校准。但是，考虑到每次相机姿势更改都需要进行新的校准，这些方法通常很麻烦且冗长。在这项工作中，我们提出了一种无标记的在线方法，用于对多个智能边缘传感器进行外部校准，仅依赖于2D人关键点检测，这些检测是在RGB摄像机图像中本地计算出的。我们的方法假定要知道的固有摄像头参数，并且需要对相机姿势进行粗略的初始估计进行启动。从中央后端收到了来自多个视图的人关键点检测，并将其同步，过滤和分配给人假设。我们使用这些人假设以因子图的形式反复解决优化问题。考虑到对遍及场景的一个或多个人的合适观察，估计的相机姿势在几分钟内将相干的外部校准汇聚在一起。我们在现实世界中评估了我们的方法，并证明与使用传统校准目标通过离线方法生成的参考校准相比，使用我们方法的校准实现了较低的再投影错误。

近年来，无人驾驶汽车（UAV）用于众多检查和视频捕获任务。但是，在障碍附近手动控制无人机是具有挑战性的，并且构成了高风险。即使对于自动飞行，全球导航计划也可能太慢，无法应对新感知的障碍。诸如风之类的干扰可能会导致与计划中的轨迹偏离。在这项工作中，我们提出了一种快速的预测障碍方法，该方法不取决于更高级别的本地化或映射，并保持无人机的动态飞行功能。它直接在LIDAR范围内实时运行，并通过计算范围图像内的角电位字段来调整当前飞行方向。随后根据轨迹预测和接触时间估计来确定速度幅度。使用硬件式模拟评估我们的方法。它可以使无人机保持安全距离，同时允许比以前直接在传感器数据上运行的反应性障碍物方法更高的飞行速度。

我们提出了一种基于直接质心控制的人形机器人的运动和平衡的综合方法。我们的方法使用人形生物的五质量描述。它从机器人的所需脚部轨迹和质心参数产生全身运动。一组简化的模型用于制定一般和直观的控制定律，然后实时应用它们，以估算和调节质量位置的中心和多体惯性主轴的方向。所提出的算法的组合产生了一条伸展的步态，并具有自然的上身运动。由于仅需要6轴IMU和关节编码器才能实现，因此机器人之间的可移植性很高。我们的方法已通过类人类开放式平台对实验进行了实验验证，证明了全身运动和推动排斥能力。

我们提出了神经链，这是一个新颖的学习框架，用于对多视图图像输入进行准确的头发几何形状和外观进行建模。从任何观点都具有高保真视图依赖性效果，可以实时渲染学习的头发模型。我们的模型可实现直观的形状和风格控制，与体积同行不同。为了实现这些特性，我们提出了一种基于神经头皮纹理的新型头发表示，该神经头皮纹理编码每个Texel位置的单个链的几何形状和外观。此外，我们基于学习的头发链的栅格化引入了一个新型的神经渲染框架。我们的神经渲染是链的和抗氧化的，使渲染视图一致且逼真。将外观与多视图几何事先结合在一起，我们首次启用了外观的联合学习和从多视图设置的显式头发几何形状。我们证明了我们的方法在各种发型的忠诚度和效率方面的功效。

深度学习模型在机器人技术中的有用性在很大程度上取决于培训数据的可用性。培训数据的手动注释通常是不可行的。合成数据是可行的替代方法，但遭受了域间隙。我们提出了一种多步方法，以获取训练数据而无需手动注释：从3D对象网格中，我们使用现代合成管道生成图像。我们利用一种最先进的图像到图像翻译方法来使合成图像适应真实域，以学习的方式最大程度地减少域间隙。翻译网络是从未配对的图像中训练的，即仅需要未经通知的真实图像集合。然后，生成和精致的图像可用于训练深度学习模型以完成特定任务。我们还建议并评估翻译方法的扩展，以进一步提高性能，例如基于补丁的训练，从而缩短了训练时间并增加了全球一致性。我们评估我们的方法并证明其在两个机器人数据集上的有效性。我们终于深入了解了博学的改进操作。

Autonomous systems not only need to understand their current environment, but should also be able to predict future actions conditioned on past states, for instance based on captured camera frames. However, existing models mainly focus on forecasting future video frames for short time-horizons, hence being of limited use for long-term action planning. We propose Multi-Scale Hierarchical Prediction (MSPred), a novel video prediction model able to simultaneously forecast future possible outcomes of different levels of granularity at different spatio-temporal scales. By combining spatial and temporal downsampling, MSPred efficiently predicts abstract representations such as human poses or locations over long time horizons, while still maintaining a competitive performance for video frame prediction. In our experiments, we demonstrate that MSPred accurately predicts future video frames as well as high-level representations (e.g. keypoints or semantics) on bin-picking and action recognition datasets, while consistently outperforming popular approaches for future frame prediction. Furthermore, we ablate different modules and design choices in MSPred, experimentally validating that combining features of different spatial and temporal granularity leads to a superior performance. Code and models to reproduce our experiments can be found in https://github.com/AIS-Bonn/MSPred.

Mohamed Bin Zayed国际机器人挑战（MBZIRC）2020为无人机（无人机）构成了不同的挑战。我们提供了四个量身定制的无人机，专门为MBZIRC的单独空中机器人任务开发，包括自定义硬件和软件组件。在挑战1中，使用高效率，车载对象检测管道进行目标UAV，以捕获来自目标UAV的球。第二个UAV使用类似的检测方法来查找和流行散落在整个竞技场的气球。对于挑战2，我们展示了一种能够自主空中操作的更大的无人机：从相机图像找到并跟踪砖。随后，将它们接近，挑选，运输并放在墙上。最后，在挑战3中，我们的UAV自动发现使用LIDAR和热敏摄像机的火灾。它用船上灭火器熄灭火灾。虽然每个机器人都具有任务特定的子系统，但所有无人机都依赖于为该特定和未来竞争开发的标准软件堆栈。我们介绍了我们最开源的软件解决方案，包括系统配置，监控，强大无线通信，高级控制和敏捷轨迹生成的工具。为了解决MBZirc 2020任务，我们在多个研究领域提出了机器视觉和轨迹生成的多个研究领域。我们介绍了我们的科学贡献，这些贡献构成了我们的算法和系统的基础，并分析了在阿布扎比的MBZIRC竞赛2020年的结果，我们的系统在大挑战中达到了第二名。此外，我们讨论了我们参与这种复杂的机器人挑战的经验教训。